Image data alteration detection device, image data alteration detection method, and data structure of image data

ABSTRACT

To provide an image data alteration detection device, an image data alteration detection method, and a data structure of image data that can easily detect an alteration in image data. Metadata in image data includes a first hash value calculated based on predetermined data, a second hash value calculated based on a character string of a script, and the script. An image alteration detection device calculates a third hash value based on the predetermined data and the script included in the metadata, calculates a fourth hash value based on a character string of the script included in the metadata, compares the first hash value included in the metadata with the calculated third hash value and the second hash value included in the metadata with the calculated fourth hash value, and detects that the image data is altered when one of the values is not equal.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of PCT International Application No.PCT/JP2018/028375 filed on Jul. 30, 2018 which claims the benefit ofpriority from Japanese Patent Application No. 2017-160606 filed on Aug.23, 2017, the entire contents of which are incorporated herein byreference.

BACKGROUND 1. Technical Field

The present disclosure relates to an image data alteration detectiondevice, an image data alteration detection method, and a data structureof image data.

2. Description of the Related Art

Changes and alterations made on data are a serious threat to devicesthat handle the data. Thus, ensuring data integrity has become animportant issue. In recent years, various technologies have beendeveloped to detect data alterations (for example, Japanese Patent No.4482451, Japanese Patent No. 5350782, and Japanese Laid-open PatentPublication No. 2007-318412 A).

In a method for controlling a user data object disclosed in JapanesePatent No. 4482451, indestructibility of a user data object is estimatedby a checksum in the package (paragraph 0015 in Japanese Patent No.5350782).

In an alteration detection device of metadata disclosed in JapanesePatent No. 5350782, an alteration is detected by calculating a hashvalue (paragraph 0049 in Japanese Patent No. 5350782).

In an image recording device disclosed in Japanese Laid-open PatentPublication No. 2007-318412 A, an alteration is determined by generatinga hash value on the basis of image data for each field (paragraph 0022in Japanese Laid-open Patent Publication No. 2007-318412 A).

In methods disclosed in Japanese Patent No. 4482451, Japanese Patent No.5350782, and Japanese Laid-open Patent Publication No. 2007-318412, thecalculation method of a hash value needs to be unified in advance, atthe data generation side and the data reproduction side. Thus, when datais altered by a third party, the unified calculation method needs to bechanged every time the data is altered. Consequently, a large amount oflabor and time is required.

Moreover, because the calculation method is defined by the type of dataand format, there is no flexibility in the calculation method of a hashvalue at the data generation side. Consequently, a large amount of datamay be altered by a specific alteration method.

SUMMARY

It is an object of the present disclosure to at least partially solvethe problems in the conventional technology.

To solve the above problems, a data alteration detection deviceaccording to the present disclosure includes an alteration detectionunit that detects an alteration in image data including data obtained byencoding an image and metadata that is data relating to the data. Themetadata at least includes a first hash value calculated based on aparameter embedded in a predetermined area of the image data in advance,a second hash value calculated based on a character string of a scriptfor generating a hash value, and the script, and the alterationdetection unit calculates a third hash value based on the parameter andthe script included in the metadata, calculates a fourth hash valuebased on the character string of the script included in the metadata,compares the first hash value included in the metadata with the thirdhash value being calculated and the second hash value included in themetadata with the fourth hash value being calculated, and detects thatthe image data is altered when one of the values is not equal.

The data alteration detection device according to the presentdisclosure, further preferably includes an image data reproduction unitthat reproduces an image based on the image data, when an alteration inthe image data is not detected by the alteration detection unit.

Further, a data alteration detection method according to the presentdisclosure includes detecting an alteration in image data including dataobtained by encoding an image and metadata that is data relating to thedata. The metadata at least includes a first hash value calculated basedon a parameter embedded in a predetermined area of the image data inadvance, a second hash value calculated based on a character string of ascript for generating a hash value, and the script. The detecting analteration further includes calculating a third hash value based on theparameter and the script included in the metadata, calculating a fourthhash value based on a character string of the script included in themetadata, and comparing the first hash value included in the metadatawith the third hash value being calculated, and the second hash valueincluded in the metadata with the fourth hash value being calculated.The comparing detects that the image data is altered when one of thevalues is not equal.

Further, a data structure of image data according to other aspect of thepresent disclosure, includes data obtained by encoding an image, andmetadata that is data relating to the data. The metadata at leastincludes a first hash value calculated based on a parameter embedded ina predetermined area of the image data in advance, a second hash valuecalculated based on a character string of a script for generating a hashvalue, and the script.

The above and other objects, features, advantages and technical andindustrial significance of this disclosure will be better understood byreading the following detailed description of presently preferredembodiments of the disclosure, when considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an overview of a generation process ofimage data in which metadata is stored;

FIG. 2 is a diagram illustrating a format example of image data;

FIG. 3 is a diagram illustrating an example in which a musical score isdisplayed in a trimmed manner according to a reproduction elapsed time;

FIG. 4 is a block diagram illustrating an example of a hardwareconfiguration of an information processing device;

FIG. 5 is a diagram illustrating an example of a functional blockconfiguration of an information processing device that executes an imagereproduction process involving a trimming process;

FIG. 6 is a flowchart illustrating a trimmed image display process;

FIG. 7 is a diagram illustrating an example of displaying subtitles oflyrics according to a reproduction elapsed time;

FIG. 8 is a diagram illustrating an example of a functional blockconfiguration of an information processing device that executes an imagereproduction process with reproduction of audio;

FIG. 9 is a flowchart illustrating a subtitle display process;

FIG. 10 is a diagram illustrating an embodiment of the presentdisclosure, and is a diagram illustrating an example of image data inwhich alteration detection data is described in metadata;

FIG. 11 is a diagram illustrating the embodiment of the presentdisclosure, and is a diagram illustrating an example of a functionalblock configuration of an information processing device that executes anexample of an image reproduction process involving detection of analteration;

FIG. 12 is a flowchart illustrating an alteration detection process;

FIG. 13 is a diagram illustrating a processing example, when theoriginal image is a map image, and when metadata includes a characterstring such as name of place that is selectively displayed according tothe position on the map and the set language;

FIG. 14 is a diagram illustrating a processing example, when theoriginal image is a photograph, and when metadata includes a characterstring such as address or name of facility of a photographing locationof the photograph;

FIG. 15 is a diagram illustrating a processing example, when theoriginal image is an image of a road guide sign, and when metadataincludes text data indicating the contents of the road guide sign;

FIG. 16 is a diagram illustrating a processing example, when theoriginal image data is encrypted by a public key, and when metadatastores the public key;

FIG. 17 is a diagram illustrating a processing example, when theoriginal image is a landscape photograph, and when metadata includesobject information such as positional information of a building and thelike in the photograph; and

FIG. 18 is a diagram illustrating a processing example, when theoriginal image is a landscape photograph, and when metadata includesobject information such as positional information of a building and thelike in the photograph.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an image reproduction technology relating to the presentdisclosure will be described in detail with reference to FIG. 1 to FIG.9, and FIG. 13 to FIG. 18. Moreover, an embodiment of an image dataalteration detection device, an image data alteration detection method,and a data structure of image data according to the present disclosurewill be described in detail with reference to FIG. 10 to FIG. 12. Eachof the image reproduction processes in FIG. 1 to FIG. 9 and FIG. 13 toFIG. 18 may be carried out by combining with an image data alterationdetection described with reference to FIG. 10 to FIG. 12. The image dataalteration detection device, the image data alteration detection method,and the data structure of image data according to the present disclosureare not limited to the disclosed embodiment. Descriptions will be madein the following order.

1. Overview of Generation Process of Image Data in which Metadata isStored

2. Example of Image Reproduction Process 3. Example of ImageReproduction Process Accompanied by Reproduction of Audio 4. Example ofImage Reproduction Process Accompanied by Detection of Alteration inImage Data 5. Modification

Overview of Generation Process of Image Data in which Metadata is Stored

FIG. 1 is a diagram illustrating an overview of a generation process ofimage data in which metadata is stored. For example, an informationprocessing device 1 of the present embodiment is a device such as alaptop computer and a desktop computer. The information processingdevice 1 has a function as an image data generation device and afunction as an image reproduction device. The image data generationdevice generates metadata and generates image data in which thegenerated metadata is stored. The image reproduction device canreproduce an image from the image data in which the metadata is stored.Thus, a computer program to cause the information processing device 1 tofunction as the image data generation device and a computer program tocause the information processing device 1 to function as the imagereproduction device are installed in the information processing device 1in advance. However, the information processing device 1 may alsofunction only as the image data generation device or only as the imagereproduction device.

The information processing device 1 inputs original image data capturedby a camera, or original image data created by image processing (alsoincludes what is called artwork that is data created by image processingsoftware), and inputs reproduction control data of the original imagedata. For example, the reproduction control data is data includingtrimming information in which time information and area information areformed as a set. The area information is information to specify acertain area in the original image data. For example, the areainformation is information including upper left coordinates, width, andheight, or information including upper left coordinates and lower rightcoordinates. The time information is information indicating an elapsedtime (elapsed period) from the start of reproduction of the originalimage data.

The information processing device 1 performs a predetermined encodingprocess on the input original image data, generates metadata from theinput reproduction control data, and generates image data including theencoded data and the generated metadata.

FIG. 2 is a diagram illustrating a format example of image data. Asillustrated in FIG. 2, image data P includes areas of start of image(SOI), application marker segment 1 (APP1), . . . , application markersegment 11 (APP11), original image data, and end of image (EOI). Forexample, the image data P in the present embodiment is defined by JPEGXT Part 3 Box file format that is an extension of a conventional jointphotographic experts group (JPEG) standards, and is specified by anextensible box-based file format allowing the user to write flexibly.

SOI is a marker at the beginning of a JPEG file, and indicates thestarting point of the JPEG file. The JPEG file is identified by readingthe SOI.

APP1 stores Exchangeable image file format (Exif).

APP11 stores metadata defined in JPEG XT Part 3 Box file format that isdescribed using JavaScript Object Notation (JSON). More specifically,APP11 stores the length of the application marker segment and aplurality of pieces of box data. Each box data stores box length, boxtype, metadata type, schema ID, and metadata. In the example in FIG. 2,box data in JPEG universal metadata box format (JUMBF)(0) stores data inwhich metadata type is multipurpose internet mail extensions (MIME),schema ID is APP/JSON, and metadata is JSON, and box data in (JUMBF)(1)stores data in which metadata type is Vender, schema ID is Vender/XXX,and metadata is XXX data.

Compressed image encoded data in JPEG format is stored as the originalimage data.

The EOI is a marker indicating the end of the JPEG file.

As illustrated in FIG. 2, by storing metadata that can be describedusing JSON in the box file data in APP11 of the image data P, andreading the data specified therein, it is possible to controlreproduction of an image.

Example of Image Reproduction Process for Displaying Image in TrimmedManner (Example Using Music Score) FIG. 3 is a diagram illustrating anexample in which a musical score is displayed in a trimmed manneraccording to a reproduction elapsed time. As illustrated in FIG. 3,image encoded data composed of a musical score with 12 bars is stored inthe original image data of image data P1. Metadata M1 described usingJSON is stored in an area of APP11 of the image data P1. In the metadataM1, the first row contains “‘clip’: [”, the second row contains “{”, thethird row contains “‘time’: 0,”, the fourth row contains “‘left’: 10,”,the fifth row contains “‘top’: 60,”, the sixth row contains “‘width’:400,”, the seventh row contains “‘height’: 100”, the eighth row contains“},”, the ninth row contains “{”, the tenth row contains “‘time’: 16,”,the eleventh row contains “‘left’: 10,”, the twelfth row contains“‘top’: 160,”, the thirteenth row contains “‘width’: 400,”, thefourteenth row contains “‘height’: 100”, the fifteenth row contains“},”, and the n-th row contains “]”.

“‘clip’” is information indicating to use a trimming function (clipfunction). Information described after “‘time’” indicates timeinformation, and information described after “‘left’”, “‘top’”, and“‘width’” indicates area information. In other words, the metadata M1includes trimming information in which time information and areainformation used for trimming a predetermined position of an image by atrimming function are formed as a set. By reading out the metadata(trimming information) M1, the information processing device 1 can trimand sequentially display a predetermined area of an image on the basisof the area information associated with the time information,corresponding to an elapsed time from the start of reproduction of theimage data P1.

In the example in FIG. 3, when the image data P1 in which the metadataM1 such as above is stored is displayed, during a period from thedisplay start time to the first 16 seconds, an area of 100 pixels inheight and 400 pixels in width is trimmed from the position of 10 pixelsfrom left and 60 pixels from top. Consequently, as indicated by an arrowA1, an area P2 of the first four bars is displayed in a trimmed manner.

Next, during a period from when 16 seconds have elapsed from the displaystart time up to 32 seconds, an area of 100 pixels in height and 400pixels in width is trimmed, from the position of 10 pixels from left and160 pixels from top. Consequently, as indicated by an arrow A2, an areaP3 of the next four bars is displayed in a trimmed manner.

An operation of displaying the image data such as above in a trimmedmanner according to the elapsed time will be described in detail belowwith reference to a flowchart.

Example of Configuration of Information Processing Device FIG. 4 is ablock diagram illustrating an example of a hardware configuration of theinformation processing device 1. The information processing device 1includes a central processing unit (CPU) 11, a read only memory (ROM)12, a random access memory (RAM) 13, a bus 14, an input unit 15, anoutput unit 16, a storage unit 17, and a communication unit 18.

The CPU 11, the ROM 12, and the RAM 13 are connected with each other viathe bus 14. The input unit 15, the output unit 16, the storage unit 17,and the communication unit 18 are also connected to the bus 14.

The input unit 15 includes an input device such as a keyboard and amouse, and supplies various types of information to the CPU 11 via thebus 14. The output unit 16 includes an output device such as a displayand a speaker, and displays an image or reproduces audio based on aninstruction from the CPU 11. The storage unit 17 includes hard disk,nonvolatile memory, and the like. The storage unit 17 stores varioustypes of data such as image data in which metadata is stored, inaddition to a computer program executed by the CPU 11. The communicationunit 18 includes a network interface and the like, and communicates withan external device, which is not illustrated, via wireless or wiredcommunication.

FIG. 5 is an example of a functional block configuration of theinformation processing device 1 that executes an example of an imagereproduction process for trimming an image serving as an informationprocessing device 1A. The information processing device 1A includes animage data generation device 30 and an image reproduction device 40. Theimage data generation device 30 generates metadata, and generates imagedata in which the generated metadata is stored. The image reproductiondevice 40 reproduces an image on the basis of the metadata.

The image data generation device 30 includes an image encoding unit 31,a metadata generation unit 32, an image data generation unit 33, and arecord control unit 34.

The image encoding unit 31 inputs original image data captured by acamera, or original image data created by image processing, and encodesthe input original image data into JPEG XT format. The obtained imageencoded data is supplied to the image data generation unit 33.

The metadata generation unit 32 inputs reproduction control dataincluding trimming information in which time information and areainformation are formed as a set, and generates metadata defined in JPEGXT Part 3 Box file format that can be described using JSON. Thegenerated metadata is supplied to the image data generation unit 33.

The image data generation unit 33 generates image data (FIG. 2) in whichthe image encoded data supplied from the image encoding unit 31, and themetadata supplied from the metadata generation unit 32 are stored. Thegenerated image data is supplied to the record control unit 34.

The record control unit 34 supplies the image data including the imageencoded data and the metadata supplied from the image data generationunit 33 to the storage unit 17. The record control unit 34 also controlsrecording of the image data in the storage unit 17.

The image reproduction device 40 includes an analysis unit 41, an imagedecoding unit 42, an image storage unit 43, an image trimming unit 44,and an output control unit 45.

On the basis of an instruction from the input unit 15, the analysis unit41 acquires the image data from the storage unit 17, and analyzes themetadata stored in the acquired image data. The analysis unit 41 alsosupplies the image encoded data in JPEG XT format that is stored in theimage data to the image decoding unit 42. The analysis unit 41 activatesan internal timer, which is not illustrated, and controls the imagetrimming unit 44 on the basis of the time measured by the internaltimer, and the trimming information including the time information thatmatches with the time measured by the internal timer, among a pluralityof pieces of trimming information in which the time information and thearea information described in the analyzed metadata are formed as a set.In other words, the analysis unit 41 controls the image trimming unit 44so that a predetermined area of an image in the image indicated by theimage data stored in the image storage unit 43 is sequentially trimmedat a prescribed timing, on the basis of the pieces of trimminginformation described in the metadata.

The image decoding unit 42 decodes the image encoded data in JPEG XTformat supplied from the analysis unit 41. The obtained image decodeddata is supplied to the image storage unit 43, and is temporality storedtherein.

In the image decoded data stored in the image storage unit 43, the imagetrimming unit 44 trims a predetermined area of the image at a prescribedtiming, on the basis of control performed by the analysis unit 41, andsupplies decoded data that corresponds to the trimming image to theoutput control unit 45.

The output control unit 45 outputs (displays) the decoded data of apredetermined area of the image supplied from the image trimming unit 44onto the display.

Operation of Information Processing Device in Example of ImageReproduction Process for Trimming Image

A trimmed image display process of the information processing device 1Awill now be described with reference to the flowchart in FIG. 6.

At step S1, the analysis unit 41 acquires image data from the storageunit 17, on the basis of an instruction from the input unit 15. At stepS2, the analysis unit 41 analyzes the metadata stored in the image data,and supplies the image encoded data in JPEG XT format that is stored inthe read image data to the image decoding unit 42.

At step S3, the image decoding unit 42 decodes the image encoded datasupplied from the analysis unit 41, and acquires image decoded data. Theimage decoded data is supplied to the image storage unit 43, and istemporarily stored therein.

At step S4, the analysis unit 41 activates the internal timer. At stepS5, the analysis unit 41 determines whether there is the time measuredby the internal timer, and the trimming information including the timeinformation that matches with the time measured by the internal timer,among a plurality of pieces of trimming information described in theanalyzed metadata.

At step S5, when it is determined that there is trimming informationincluding the time information that matches with the time measured bythe internal timer (Yes at step S5), the analysis unit 41 controls theimage trimming unit 44 on the basis of the trimming informationincluding the time information that matches with the time measured bythe internal timer.

At step S6, in the image decoded data stored in the image storage unit43, the image trimming unit 44 retrieves image decoded data thatcorresponds to a predetermined area of the image on the basis of thearea information associated with the time information, on the basis ofcontrol performed by the analysis unit 41, and supplies the data to theoutput control unit 45.

At step S7, the output control unit 45 outputs the image decoded datathat corresponds to the predetermined area of the image supplied fromthe image trimming unit 44 onto the display. Then, the process returnsto step S5, and the process described above is repeated until it isdetermined that there is no trimming information including the timeinformation that matches with the time measured by the internal timer.

Then, at step S5, when it is determined that there is no trimminginformation including the time information that matches with the timemeasured by the internal timer (No at step S5), the trimmed imagedisplay process illustrated in FIG. 6 will be finished.

In this manner, with the information processing device 1A illustrated inFIG. 5, it is possible to trim and sequentially display only apredetermined area of an image, by generating image data including dataobtained by encoding an image and metadata that at least includestrimming information in which time information and area information areformed as a set, and on the basis of the area information associatedwith the time information, at a display timing that matches with thetime information described in the metadata. Thus, the data control canbe simplified, because the image data includes a display timing and datato be controlled. Moreover, the image area to be displayed and thereproduction timing of the image area can be easily changed by justediting the information in the metadata, without using a specific deviceor software. Consequently, it is possible to easily display an imagecorresponding to the reproduction elapsed time.

Moreover, the information processing device 1A may also include an audiodata reproduction unit, and cause the storage unit 17 to store audiodata in association with the image data. With such a configuration, whenthe image data is displayed, the information processing device 1A canreproduce the audio data associated with the image data. For example,when musical score data for piano is displayed, the informationprocessing device 1A can also reproduce audio data of the pianoperformance, which will be a guide for the musical score. Consequently,the user can practice piano according to the guide performance.Moreover, when musical score data for piano is displayed, theinformation processing device 1A can reproduce audio data of violinperformance on the basis of the musical score at the same time.Consequently, the user can enjoy playing duet with violin, by onlyplaying piano.

Moreover, in the above, the information processing device 1A may alsodescribe animation information in metadata that at least includestrimming information in which time information and area information areformed as a set. With such a configuration, when the image data isdisplayed, the information processing device 1A can also display animage based on the animation information associated with image data. Forexample, when a predetermined area of musical score data for piano isdisplayed, the information processing device 1A can also display animage of a guiding function (animation indicating which key to bepressed next) that allows the user to play the musical score with pianoin a superposed manner. Consequently, the user can practice piano usingthe guiding function.

Example of Image Reproduction Process Accompanied by Reproduction ofAudio

(Example Using Lyrics Data)

FIG. 7 is a diagram illustrating an example of displaying subtitles oflyrics data according to a reproduction elapsed time of audio data. Asillustrated in FIG. 7, image encoded data composed of artwork is storedin the original image data of image data P11. Metadata M11 describedusing JSON is stored in an area of APP11 of the image data P11. In themetadata M11, the first row contains “‘lyrics’: [”, the second rowcontains “{”, the third row contains “‘time’: 58”, the fourth rowcontains “‘text’: ‘Oh, Kanazawa is ˜’”, the fifth row contains “},”, thesixth row contains “{”, the seventh row contains “‘time’: 65”, theeighth row contains “‘text’: ‘snowing again today ˜’”, the ninth rowcontains “},”, and the n-th row contains “]”.

“‘lyrics’” is information indicating to use a lyrics display function.Information described after “‘time’” indicates time information, andinformation described after “‘text’” indicates text data. In otherwords, the metadata M11 includes subtitle information in which timeinformation and text data for displaying lyrics by the lyrics displayfunction are formed as a set. By generating audio data embedded with theimage data P11 in which the metadata M11 is stored, when the audio datais reproduced, the information processing device 1 can acquire the imagedata P11 embedded in the audio data. Moreover, by reading the metadata(subtitle information) M11 stored in the acquired image data P11, theinformation processing device 1 can sequentially display subtitles onthe basis of the text data associated with the time information,corresponding to an elapsed time from the start of reproduction of audiodata.

In the example in FIG. 7, when the audio data embedded with the imagedata P11 in which the metadata M11 such as above is stored isreproduced, “Oh, Kanazawa is ˜” is read out during a period from when 58seconds have passed since the reproduction start time up to 65 seconds.Consequently, as indicated by an arrow A11, a text of “Oh, Kanazawa is˜” is displayed on an image P12 as subtitles in a superposed manner.

Then, “snowing again today ˜” is read out during a period from when 65seconds have passed since the reproduction start time up to the nexttime information. Consequently, as indicated by an arrow A12, a text of“snowing again today ˜” is displayed on an image P13 as subtitles in asuperposed manner.

Details of an operation of displaying subtitles according to thereproduction elapsed time of audio data such as above will be describedin detail below with reference to a flowchart.

Example of Functional Configuration of Information Processing Devicethat Executes Example of Image Reproduction Process Accompanied byReproduction of Audio

A hardware configuration of an information processing device thatexecutes an example of an image reproduction process accompanied byreproduction of audio described above is the same as that illustrated inFIG. 4. Consequently, the explanation thereof will be omitted. FIG. 8 isan example of a functional block configuration of the informationprocessing device 1 that executes an example of an image reproductionprocess accompanied by reproduction of audio, serving as an informationprocessing device 1B. The information processing device 1B includes adata generation device 50 and an audiovisual player 60. The datageneration device 50 generates metadata, generates image data in whichthe generated metadata is stored, and generates audio data in which thegenerated image data is embedded. The audiovisual player 60 reproducesaudio from the audio data, and reproduces an image from the image dataon the basis of the metadata.

The data generation device 50 includes an image encoding unit 51, ametadata generation unit 52, a data generation unit 53, and a recordcontrol unit 54.

The image encoding unit 51 inputs original image data captured by acamera or original image data created by image processing, and encodesthe input original image data into JPEG XT format. The encoded data issupplied to the data generation unit 53.

The metadata generation unit 52 inputs reproduction control dataincluding subtitle information in which time information and text dataare formed as a set, and generates metadata defined in JPEG XT Part 3Box file format that can be described using JSON. The generated metadatais supplied to the data generation unit 53.

The data generation unit 53 generates image data (FIG. 2) in which theencoded data supplied from the image encoding unit 51, and the metadatasupplied from the metadata generation unit 52 are stored. The datageneration unit 53 externally inputs audio data, embeds the image datain which the metadata is stored in the input audio data, and suppliesthe data to the record control unit 54.

The record control unit 54 supplies the audio data that is supplied fromthe data generation unit 53, and in which the image data including theimage encoded data and the metadata is embedded, to the storage unit 17.The record control unit 54 also controls recording of the audio data inthe storage unit 17.

The audiovisual player 60 includes an analysis unit 61, an imagedecoding unit 62, a text drawing unit 63, and an output control unit 64.

The analysis unit 61 acquires audio data from the storage unit 17 on thebasis of an instruction from the input unit 15, and supplies theacquired audio data to the output control unit 64. The analysis unit 61also acquires image data embedded in the acquired audio data, andanalyzes the metadata stored in the acquired image data. By theanalysis, the image encoded data in JPEG XT format stored in the imagedata is supplied to the image decoding unit 62.

Moreover, the analysis unit 61 activates the internal timer, which isnot illustrated, and controls the text drawing unit 63 on the basis ofthe time measured by the internal timer, and the subtitle informationincluding the time information that matches with the time measured bythe internal timer, among a plurality of pieces of subtitle informationin which the time information and the text data described in theanalyzed metadata are formed as a set. In other words, the analysis unit61 controls the text drawing unit 63 so that the text data issequentially turned into an image at a prescribed timing, on the basisof the pieces of subtitle information described in the metadata.

The image decoding unit 62 decodes the image encoded data in JPEG XTformat supplied from the analysis unit 61. The decoded image data issupplied to the output control unit 64.

On the basis of control performed by the analysis unit 61, the textdrawing unit 63 turns the text data supplied from the analysis unit 61into image data at a prescribed timing, and supplies the data to theoutput control unit 64.

The output control unit 64 outputs and reproduces audio on the basis ofthe audio data supplied from the analysis unit 61 to the speaker, andsuperposes the image data supplied from the text drawing unit 63 on theimage data supplied from the image decoding unit 62, and outputs(displays) the data onto the display.

Operation of Information Processing Device in Example of ImageReproduction Process Accompanied by Reproduction of Audio

A subtitle display process of the information processing device 1B willnow be described with reference to the flowchart in FIG. 9.

At step S11, on the basis of an instruction from the input unit 15, theanalysis unit 61 acquires audio data from the storage unit 17. At stepS12, the analysis unit 61 analyzes metadata of the image data embeddedin the audio data. The acquired audio data is supplied to the outputcontrol unit 64, and the image encoded data in JPEG XT format stored inthe analyzed metadata is supplied to the image decoding unit 62.

At step S13, the image decoding unit 62 generates image decoded data bydecoding the image encoded data in JPEG XT format supplied from theanalysis unit 61, and supplies the data to the output control unit 64.At step S14, the output control unit 64 outputs and reproduces audio onthe basis of the audio data to the speaker.

At step S15, the analysis unit 61 activates the internal timer. At stepS16, the analysis unit 61 determines whether there is the time measuredby the internal timer, and the subtitle information including the timeinformation that matches with the time measured by the internal timer,among the pieces of subtitle information described in the analyzedmetadata.

At step S16, when it is determined that there is subtitle informationincluding the time information that matches with the time measured bythe internal timer (Yes at step S16), the analysis unit 61 controls thetext drawing unit 63 on the basis of the subtitle information includingthe time information that matches with the time measured by the internaltimer.

At step S17, on the basis of control performed by the analysis unit 61,the text drawing unit 63 turns the text data associated with the timeinformation into image data, and supplies the data to the output controlunit 64.

At step S18, the output control unit 64 superposes the text image datasupplied from the text drawing unit 63 on the image data supplied fromthe image decoding unit 62, and displays and outputs the data. Then, theprocess returns to step S16, and the process described above isrepeated, until it is determined that there is no subtitle informationincluding the time information that matches with the time measured bythe internal timer.

Then, at step S16, when it is determined that there is no subtitleinformation including the time information that matches with the timemeasured by the internal timer (No at step S16), the subtitle displayprocess illustrated in FIG. 9 will be finished.

As described above, by generating audio data embedded with the imagedata including the encoded data and the metadata that at least includesthe subtitle information in which time information and text data areformed as a set, when the audio data is reproduced, and at the displaytiming that matches with the time information described in the metadataof the image data embedded in the audio data, it is possible to turn thetext data associated with the time information into image data, anddisplay subtitles by superposing the obtained text image data on theimage data.

Consequently, it is possible to easily display an image corresponding tothe reproduction elapsed time of the audio data. Moreover, for example,because the image data, audio data, and text data described above can becontrolled as a single music file, it is possible to easily handle data.Furthermore, because the subtitle information is stored in the textdata, it is possible to easily edit subtitle time information.

In the above, the information processing device 1B may also describecolor information, font information, information indicating the presenceof shadow of the text, background color information, and the like on themetadata that at least includes the subtitle information in which timeinformation and text data are formed as a set. With such aconfiguration, the information processing device 1B may not only displaysimple subtitles but also display visually enjoyable subtitles, whensubtitles are displayed.

Example of Image Reproduction Process Accompanied by Detection ofAlteration

FIG. 10 is a diagram illustrating an embodiment of the presentdisclosure, and is a diagram illustrating an example of image data inwhich alteration detection data is described in metadata. As illustratedin FIG. 10, image encoded data in which the original image is aphotograph is stored in the original image data of image data P21.Metadata M21 described using JSON is stored in an area of APP11 of theimage data P21. A hash value A, a hash value B, and script are describedin the metadata M21. The hash value A is a value obtained when thescript is executed using seed data as an argument. The seed data is data(parameter) embedded in a predetermined area of the image data P21 inadvance. The hash value B is a value obtained when the script isexecuted using a program character string in the script as an argument.The script is a hash function (computer program) for calculating a hashvalue. In other words, data for detecting alteration is described in themetadata M21, and the information processing device 1 can detectalteration of the image data P21, by reading the metadata (alterationdetection data) M21 and executing the script.

Details of an operation of reading and executing the alterationdetection data such as above will be described below with reference to aflowchart.

Example of Functional Configuration of Information Processing Devicethat Executes Example of Image Reproduction Process Accompanied byDetection of Alteration

A hardware configuration of an information processing device thatexecutes an example of an image reproduction process accompanied bydetection of alteration is the same as that illustrated in FIG. 4.Consequently, the explanation thereof will be omitted. FIG. 11 is anexample of a functional block configuration of the informationprocessing device 1 that executes an example of an image reproductionprocess accompanied by detection of alteration serving as an informationprocessing device 1C. In FIG. 11, the same reference numerals denote thecomponents the same as those in FIG. 5, and the repeated descriptionthereof are omitted as appropriate. The information processing device 1Cincludes the image data generation device 30 and an image dataalteration detection device 70. The image data generation device 30generates metadata, and generates image data in which the generatedmetadata is stored. The image data alteration detection device 70detects whether the image data in which the metadata is stored isaltered, and reproduces the image data when the image data is notaltered.

The metadata generation unit 32 inputs reproduction control dataincluding the hash value A, the hash value B, and the script fordetecting alteration, and generates metadata defined in JPEG XT Part 3Box file format that can be described using JSON. The generated metadatais supplied to the image data generation unit 33.

The image data alteration detection device 70 includes an analysis unit71, a comparison unit 72, an alteration detection unit 73, an imagedecoding unit 74, and an output control unit 75.

On the basis of an instruction from the input unit 15, the analysis unit71 acquires image data from the storage unit 17, analyzes the metadatastored in the acquired image data, and supplies the alteration detectiondata (hash value A, hash value B, and script) described in the metadatato the comparison unit 72. The analysis unit 71 also supplies theencoded data in JPEG XT image format stored in the image data to theimage decoding unit 74. The analysis unit 71 reads out the seed dataembedded in the image data using a predetermined method, and suppliesthe data to the comparison unit 72.

The comparison unit 72 calculates a hash value A′ on the basis of thescript and the seed data included in the alteration detection datasupplied from the analysis unit 71, and compares the calculated hashvalue A′ with the hash value A described in the metadata (alterationdetection data). Moreover, the comparison unit 72 calculates a hashvalue B′ on the basis of the program character string in the scriptincluded in the alteration detection data, and compares the calculatedhash value B′ with the hash value B described in the metadata(alteration detection data). The comparison results are supplied to thealteration detection unit 73.

On the basis of the two comparison results of the comparison unit 72,the alteration detection unit 73 detects whether the image data isaltered, and when it is determined that the image data is not altered(the hash value A and the hash value B are both correct), the alterationdetection unit 73 causes the image decoding unit 74 to execute adecoding process. When it is detected that the image data is altered(one or both of the hash value A and the hash value B are not correct),the alteration detection unit 73 prohibits the image decoding unit 74 toexecute the decoding process.

On the basis of control performed by the alteration detection unit 73,when an execution of the decoding process is instructed, the imagedecoding unit 74 decodes the image encoded data in JPEG XT formatsupplied from the analysis unit 71, and supplies the data to the outputcontrol unit 75 as the image decoded data. On the basis of controlperformed by the alteration detection unit 73, when the decoding processis prohibited, the image decoding unit 74 does not decode the imageencoded data in JPEG XT format supplied from the analysis unit 71, andsupplies the data to the output control unit 75.

The output control unit 75 outputs (displays) the data supplied from theimage decoding unit 74 onto the display.

Operation of Information Processing Device in Example of ImageReproduction Process Accompanied by Detection of Alteration

An alteration detection process of the information processing device 1Cin an example of the image reproduction process accompanied by detectionof alteration having the configuration such as above will be describedwith reference to the flowchart in FIG. 12.

At step S21, the analysis unit 71 acquires image data from the storageunit 17, on the basis of an instruction from the input unit 15. At stepS22, the analysis unit 71 analyzes the metadata stored in the imagedata, and supplies the alteration detection data (hash value A, hashvalue B, and script) described in the metadata to the comparison unit72. The analysis unit 71 also supplies the image encoded data in JPEG XTformat stored in the read image data to the image decoding unit 74.Moreover, the analysis unit 71 reads out the seed data embedded in theimage data using a predetermined method, and supplies the data to thecomparison unit 72.

At step S23, the comparison unit 72 executes the script described in themetadata (alteration detection data) using the seed data supplied fromthe analysis unit 71 as an argument, and calculates the hash value A′.At step S24, the comparison unit 72 compares the hash value A describedin the metadata (alteration detection data) with the calculated hashvalue A′.

At step S25, the comparison unit 72 executes the script using theprogram character string in the script described in the metadata(alteration detection data) as an argument, and calculates the hashvalue B′. At step S26, the comparison unit 72 compares the hash value Bdescribed in the metadata (alteration detection data) with thecalculated hash value B′. The comparison results at step S24 and stepS26 are supplied to the alteration detection unit 73.

At step S27, the alteration detection unit 73 determines whether theimage data is altered from the two comparison results. When one or bothof the comparison results are different, the alteration detection unit73 determines that the image data is altered (Yes at step S27), and atstep S28, prohibits the decoding process of the image decoding unit 74.Consequently, the image decoding unit 74 does not decode the imageencoded data in JPEG XT format supplied from the analysis unit 71, andsupplies the data to the output control unit 75. The output control unit75 then outputs (displays) the data supplied from the image decodingunit 74 onto the display.

At step S27, when the two comparison results are the same, thealteration detection unit 73 determines that the image data is notaltered (No at step S27). Then, at step S29, the alteration detectionunit 73 causes the image decoding unit 74 to execute the decodingprocess. The image decoding unit 74 decodes the image encoded data inJPEG XT format supplied from the analysis unit 71, and supplies the datato the output control unit 75 as the image decoded data. The outputcontrol unit 75 outputs (displays) the image decoded data supplied fromthe image decoding unit 74 onto the display.

As described above, by generating image data including the encoded dataand the metadata that at least includes the alteration detection data,reading out the alteration detection data described in the metadata, andexecuting the script, it is possible to easily detect whether the imagedata is altered. When it is determined that the image data is altered,it is possible to prohibit the decoding process. Consequently, comparedto the conventional alteration detection method using the hash value,the script for calculating the hash value is supplied with the imagedata. Thus, it is possible to easily detect alteration. Moreover, whenalteration is attempted, because the calculation method of a hash valuecan be changed for each image data, it is difficult to uniquely alterthe image data. Thus, it is not possible to establish the alterationmethod. Moreover, it is possible to easily verify the alteration made onthe image data generated by the other data provider.

In the above, the seed data is embedded in a predetermined area of theimage data P21 in advance. However, it is not limited thereto, and theseed data may be unified by the same standard that handles data or thelike, or the seed data may be stored in the metadata.

Moreover, the hash value B′ calculated at step S25 is obtained byexecuting the script by using the program character string in the scriptas an argument. However, the hash value B′ may also be obtained byexecuting the script using the program character string in the scriptand the seed data as arguments.

Modifications First Modification

The information processing devices 1A, 1B, and 1C may also generateimage data including image encoded data and metadata having a characterstring such as a place name that is selectively displayed according tothe positional information on the map and the set language.Consequently, when an image is displayed on the basis of the image data,the information processing devices 1A, 1B, and 1C can acquire thecharacter string associated with the language set in the informationprocessing devices 1A, 1B, and 1C in the metadata stored in the imagedata, and display the acquired character string at a predeterminedposition in a superposed manner.

FIG. 13 is a diagram illustrating a usage example of image dataincluding image encoded data and metadata having a character string suchas a place name that is selectively displayed according to the positionon the map and the set language.

As illustrated in FIG. 13, original image data of image data P31 storesimage encoded data in which an original image of a map of Japan isencoded. Metadata M31 described using JSON is stored in an area of APP11in the image data P31. In the metadata M31, the first row contains“‘point’: {”, the second row contains “‘Sapporo’: {”, the third rowcontains “‘x’: 560,”, the fourth row contains “‘y’: 80,”, the fifth rowcontains “‘name’: {”, the sixth row contains “‘en-US’: ‘Sapporo’,”, theseventh row contains “‘ja-JP’: ‘

’”, the eighth row contains “}”, the ninth row contains “},”, the tenthrow contains “‘Tokyo’: {”, the eleventh row contains “‘x’: 600,”, thetwelfth row contains “‘y’: 600,”, the thirteenth row contains “‘name’:{”, the fourteenth row contains “‘en-US’: ‘Tokyo’,”, the fifteenth rowcontains “‘ja-JP’: ‘

’”, the sixteenth row contains “}”, the seventeenth row contains “},”,the eighteenth row contains “‘Naha’: {”, the nineteenth row contains“‘x’: 200,”, the twentieth row contains “‘y’: 1100,”, the twenty-firstrow contains “‘name’: {”, the twenty-second row contains “‘en-US’:‘Naha’,”, the twenty-third row contains “‘ja-JP’: ‘

’”, the twenty-fourth row contains “}”, the twenty-fifth row contains“},”, and the twenty-sixth row contains “}”.

“‘point’” is information indicating to use a function of pointing aspecific position on the screen. Information described after “‘x’” and“‘y’” of “‘Sapporo’”, “‘Tokyo’”, and “‘Naha’” each indicate thecoordinate information of the place name (position) on the map.Information described after “‘name’” indicates language, and informationdescribed after “‘en-US’” indicates the place name to be displayed whenthe language is set. Moreover, information described after “‘ja-JP’”indicates the place name (character string) to be displayed when thelanguage is set. In other words, the metadata M31 includes place nameinformation in which the coordinate information, the set language, andthe place name that are used for displaying the place name in apredetermined language by a function of pointing a specific position onthe screen are formed as a set. The information processing devices 1A,1B, and 1C can display the place name according to the predeterminedlanguage set for the terminal at a predetermined position in asuperposed manner, by reading out the metadata (place name information),when the image data is displayed.

In the example in FIG. 13, when an image is displayed on the basis ofthe image data P31 in which the metadata M31 such as above is stored,and when the language of the information processing devices 1A, 1B, and1C is set to Japanese, Japanese notation of the place names (

,

,

) subsequent to “‘ja-JP’” in the metadata M31 will be read out.Consequently, as indicated by an arrow A31, the information processingdevices 1A, 1B, and 1C display the place names in Japanese onpredetermined positions of a map of Japan display P32 in a superposedmanner. Moreover, when the language of the information processingdevices 1A, 1B, and 1C is set to English, the place names (Sapporo,Tokyo, and Naha) subsequent to “‘en-US’” in the metadata M31 will beread out. Consequently, as indicated by an arrow A32, the informationprocessing devices 1A, 1B, and 1C display the place names in English onpredetermined positions of a map of Japan display P33 in a superposedmanner.

In this manner, with the first modification, by generating the imagedata including the image encoded data and the metadata having acharacter string such as a place name that is selectively displayedaccording to the positional information on the map and the set language,when an image is displayed on the basis of the image data, it ispossible to display the place names associated with the language set forthe information processing devices 1A, 1B, and 1C at predeterminedpositions in a superposed manner, on the basis of the place nameinformation described in the metadata.

Second Modification

The information processing devices 1A, 1B, and 1C may also generateimage data including image encoded data and metadata having a characterstring such as address or name of facility of the photographing locationof the image. Consequently, the information processing devices 1A, 1B,and 1C can acquire the character string in the metadata stored in theimage data, and display the acquired character string on the image in asuperposed manner, when the image is displayed. Moreover, theinformation processing devices 1A, 1B, and 1C can search an image usingthe character string in the metadata stored in the image data as asearch key.

FIG. 14 is a diagram illustrating a usage example of image dataincluding image encoded data and metadata having a character string suchas address or name of facility of a photographing location of the image.

As illustrated in FIG. 14, a photograph captured in Okinawa is encodedand stored in the original image data of image data P41, as imageencoded data. Metadata M41 described using JSON is stored in an area ofAPP11 of the image data P41. In the metadata M41, the first row contains“‘location’: {”, the second row contains “‘address’: ‘Shurikinjyocho1-2, Naha-city, Okinawa prefecture’”, and the third row contains “}”.

“‘location’” is information indicating to use a function capable ofspecifying the current position and linking with service. Informationdescribed after “‘address’” indicates the address of the photographinglocation. In other words, the metadata M41 includes informationindicating the address of the photographing location, and theinformation processing devices 1A, 1B, and 1C can display theinformation indicating the address of the photographing locationdescribed in the metadata in a superposed manner, by reading out themetadata when the image is displayed.

In the example in FIG. 14, when an image is displayed on the basis ofthe image data P41 in which the metadata M41 such as above is stored,the character string (Shurikinjyocho 1-2, Naha-city, Okinawa prefecture)subsequent to “‘address’” in the metadata M41 is read out. Consequently,as indicated by an arrow A41, the information processing devices 1A, 1B,and 1C display the address of the photographing location on an imagedisplay P42 in a superposed manner.

Moreover, as indicated by an arrow A42, the information processingdevices 1A, 1B, and 1C can supply the image data P41 in which themetadata M41 such as above is stored to database (DB) 101 connected viaa network, which is not illustrated, and control the image data P41therein. Consequently, when an image is searched using “Okinawa” as asearch key, the information processing devices 1A, 1B, and 1C can searchthe image data including “Okinawa” in the metadata M41, from a pluralityof pieces of image data controlled by the database 101. Then, asindicated by an arrow A43, the information processing devices 1A, 1B,and 1C can display an image list P43 including a plurality of thumbnailimages of the searched image data.

In this manner, with the second modification, by generating the imagedata including the image encoded data and the metadata having acharacter string such as address or name of facility of thephotographing location, when the image is displayed, the address, thename of facility, or the like of the photographing location stored inthe image data can be displayed in a superposed manner. Moreover, bycontrolling the generated image data by database, the image data inwhich the metadata including a search key is stored can be easilysearched, by specifying the search key.

Third Modification

The information processing devices 1A, 1B, and 1C may also generateimage data including image encoded data and metadata having text dataindicating the content of the image encoded data. Consequently, theinformation processing devices 1A, 1B, and 1C can acquire text data inthe metadata stored in the image data, convert the acquired text data toaudio by a text reading function, and reproduce the converted audio,when the image is displayed on the basis of the image data.

FIG. 15 is a diagram illustrating a usage example of image dataincluding image encoded data and metadata having text data indicatingthe content of the image encoded data.

As illustrated in FIG. 15, data of a navigation image to be displayed ona car navigation system is stored in the original image data of imagedata P51, as image encoded data. Metadata M51 described using JSON isstored in an area of APP11 of the image data P51. In the metadata M51,the first row contains “‘tts’: {”, the second row contains “‘lang’:‘ja-JP’,”, the third and fourth rows contain “‘text’: ‘Traffic bound forTokushima Honcho is congested. It takes about 20 minutes to TokushimaHoncho.’”, and the fifth row contains “}”.

“‘tts’” is information indicating to use a text reading function calledtext-to speech (tts) system. Information described after “‘lang’”indicates the language specified when the text reading function is used.

Information described after “‘text’” indicates the text data read outwhen the tts system is used. In other words, the metadata M51 includestext data for reading out in Japanese by the text reading function.Consequently, the information processing devices 1A, 1B, and 1C canreproduce audio on the basis of the text data described in the metadata,by reading out the metadata when the image data is displayed.

In the example in FIG. 15, when the image is displayed on the basis ofthe image data P51 in which the metadata M51 such as above is stored,the text data (Traffic bound for Tokushima Honcho is congested. It takesabout 20 minutes to Tokushima Honcho) subsequent to “‘text’” in themetadata M51 is read out. Consequently, as indicated by an arrow A51,the information processing devices 1A, 1B, and 1C can display an imageP52, and reproduce (read out) audio on the basis of the text indicatedin a balloon S51, by using the text reading function.

In this manner, with the third modification, by generating image dataincluding the image encoded data and the metadata having the text dataindicating the content of the image encoded data, when the image isdisplayed on the basis of the image data, it is possible to reproduceaudio on the basis of the text data stored in the image data.

Fourth Modification

The information processing devices 1A, 1B, and 1C may also generateimage data including image encoded data encrypted by a public key, andmetadata in which the public key is stored. Consequently, theinformation processing devices 1A, 1B, and 1C can acquire the public keyin the metadata stored in the image data, when the image is displayed,and decode and display the image encoded data, only when the informationprocessing devices 1A, 1B, and 1C have a secret key that links to theacquired public key.

FIG. 16 is a diagram illustrating a usage example of image dataincluding image encoded data encrypted by a public key, and metadata inwhich the public key is stored.

As illustrated in FIG. 16, the image encoded data encrypted by a publickey is stored in the original image data of image data P61. Metadata M61described using JSON is stored in an area of APP11 of the image dataP61. A thumbnail image P61 a in a plain text is also stored in an areaof APP1 (Exif) of the image data P61. In the metadata M61, the first rowcontains “‘encrypt’: {”, the second row contains “‘OID’:‘1.2.840.10045.2.1’,”, the third row contains “‘public_key’:‘04FC2E8B81DD . . . ’”, and the fourth row contains “}”.

“‘encrypt’” is information indicating to use the encryption function.Information described after “‘OID’” indicates information foridentifying an object, and information described after “‘public_key’”indicates a public key. In other words, the metadata M61 includes thepublic key used for encrypting the image encoded data. The informationprocessing devices 1A, 1B, and 1C can decode and display the imageencoded data in the image data P61, by reading out the metadata when theimage is displayed, and only when the information processing devices 1A,1B, and 1C have a secret key that links to the public key described inthe metadata.

In the example in FIG. 16, when an image is displayed on the basis ofthe image data P61 in which the metadata M61 such as above is stored,the public key (04FC2E8B81DD . . . ) subsequent to “‘public_key’” in themetadata M61 is read out. Consequently, when the information processingdevices 1A, 1B, and 1C include a secret key 111 that links to the readpublic key, the information processing devices 1A, 1B, and 1C decode(decipher) the image encoded data in the image data P61 using the secretkey 111, and as indicated by an arrow A61, display an image P62.

Moreover, when the information processing devices 1A, 1B, and 1C do notinclude the secret key 111 that links to the public key read out fromthe metadata M61, the information processing devices 1A, 1B, and 1Ccannot decode the image encoded data in the image data P61, and asindicated by an arrow A62, display data P63 remained encrypted.

In this manner, with the fourth modification, by generating the imagedata including the image encoded data encrypted by the public key andthe metadata in which the public key is stored, when the image isdisplayed, the information processing devices 1A, 1B, and 1C can decodeand display the encrypted image encoded data, only when the informationprocessing devices 1A, 1B, and 1C include the secret key that links tothe public key in the metadata stored in the image data.

Fifth Modification

The information processing devices 1A, 1B, and 1C may also generateimage data including image encoded data and metadata having object (forexample, facility) information identified on the basis of thephotographing position of the original image, direction, angle of view,and map information. Consequently, the information processing devices1A, 1B, and 1C can search an image using the object information of themetadata stored in the image data as a search key.

FIG. 17 and FIG. 18 are diagrams each illustrating a usage example ofimage data including image encoded data and metadata having objectinformation identified on the basis of the photographing position of theoriginal image, direction, angle of view, and map information.

As illustrated in FIG. 17, a photographed image of Tokyo tower atlatitude 35.65851 and longitude 139.745433 is encoded and stored in eachof the original image data of image data P71 and image data P72 as imageencoded data. Exif information of latitude 35.6591, longitude139.741969, and azimuth N90° is stored in an area of APP1 (Exif) of theimage data P71. Exif information of latitude 35.65851, longitude139.745433, and azimuth N315° is stored in an area of APP1 (Exif) of theimage data P72.

An operation unit 112 of the information processing devices 1A, 1B, and1C inputs the image data P71, refers to Map database 111 connected via anetwork, which is not illustrated, and acquires object informationrelating to the Exif information stored in the image data P71. On thebasis of the object information acquired from the Map database 111, asindicated by an arrow A71, the operation unit 112 generates metadata M71described using JSON.

An operation unit 113 of the information processing devices 1A, 1B, and1C inputs the image data P72, refers to the Map database 111 connectedvia a network, which is not illustrated, and acquires object informationrelating to the Exif information stored in the image data P72. On thebasis of the object information acquired from the Map database 111, asindicated by an arrow A72, the operation unit 113 generates metadata M72described using JSON.

In metadata M71 and M72, the first row contains “‘objects’: [”, thesecond row contains “{”, the third row contains “‘name’: ‘Tokyotower’,”, the n-1st row contains “}”, and the n-th row contains “]”.Information described after “‘objects’” indicates the objectinformation. In other words, the metadata M71 and M72 include the objectinformation relating to the photographing position.

The information processing devices 1A, 1B, and 1C store the generatedmetadata M71 in an area of APP11 of the image data P71, and store thegenerated metadata M72 in an area of APP11 of the image data P72.

As indicated by an arrow A81 in FIG. 18, the information processingdevices 1A, 1B, and 1C can supply the image data P71 in which themetadata M71 is stored and the image data P72 in which the M72 isstored, to object database 121 connected via a network, which is notillustrated, and control the image data P71 and the image data P72therein. Consequently, when an image is searched using “Tokyo tower” asa search key, the information processing devices 1A, 1B, and 1C cansearch the image data P71 and P72 in which the metadata M71 and M72include “Tokyo tower”, from a plurality of pieces of image datacontrolled by the database 121. Then, as indicated by an arrow A82, theinformation processing devices 1A, 1B, and 1C can display an image listP81 including a plurality of thumbnail images of the searched imagedata.

In this manner, with the fifth modification, by generating the imagedata including the encoded data and the metadata having the objectinformation identified on the basis of the photographing position of theimage data, direction, angle of view, and map information, andcontrolling the generated image data by the database, it is possible toeasily search the image data in which the metadata including a searchkey is stored, by specifying the search key.

The embodiment of the present disclosure has been described. However,the present disclosure is not limited to the embodiment described above,and various modifications may be made without departing from the spiritand scope of the present disclosure. For example, time information andarea information are described in the metadata explained in the exampleof the image reproduction process for trimming an image, timeinformation and text data are described in the metadata explained in theexample of the audio image reproduction process, and alterationdetection data is described in the metadata explained in the example ofthe image reproduction process accompanied by detection of alteration.However, it is also possible to generate metadata in which timeinformation, area information, and text information are described. Withsuch a configuration, at the display timing that matches with the timeinformation described in the metadata, it is possible to display only apredetermined area of image data in a trimmed manner, on the basis ofthe area information associated with the time information. Moreover, itis possible to turn the text data associated with the time informationinto an image, and display subtitles by superposing the text imageturned into an image on the image data.

Moreover, it is possible to generate metadata in which time information,area information, and alteration detection data are described, metadatain which time information, text data, and alteration detection data aredescribed, and metadata in which time information, area information,text information, and alteration detection data are described. With sucha configuration, only when it is detected that the image data is notaltered, according to the alteration detection data described in themetadata, it is possible to display only a predetermined area of theimage data in a trimmed manner at a prescribed display timing, displaysubtitles on the image data at a prescribed timing, or display only apredetermined area of the image data in a trimmed manner and displaysubtitles on the image data at a prescribed timing.

In the modifications, the object information, the photographing positioninformation, and the like are described in the metadata. However, it isnot limited thereto, and the image data may contain information such asthe face of Mr. Yamada is at an x coordinate of 300 and y coordinate of200, and the face of Mr. Suzuki is at an x coordinate of 500 and ycoordinate of 300. With such a configuration, it is possible to extractan image of Mr. Yamada from a plurality of pieces of image data, andsearch the face (position) of Mr. Yamada in the extracted image.

Moreover, data such as image information, date and time, place, status,and the like detected by performing a predetermined image recognitionprocess on the image data captured by a drive recorder, a securitycamera, and the like may be described in metadata. With such aconfiguration, it is possible to extract an image in a dangeroussituation from a plurality of pieces of image data, by image analysis.

In the above, the image data generation device 30, the imagereproduction device 40, the audio image data generation device 50, theaudiovisual player 60, and the image data alteration detection device 70are provided in the same information processing devices 1A, 1B, and 1C.However, each of the functions may be provided as a separate device.

Moreover, the series of processes described above may be executed byhardware or may be executed by software. When the series of processes isto be executed by software, a computer program that configures thesoftware is installed into a computer embedded in dedicated hardware, orinto a general personal computer, for example, capable of executingvarious functions by installing various computer programs, from acomputer program recording medium.

The computer program executed by the computer may be a computer programthat performs processing in time series according to the order describedin the present specification, or may be a computer program that performsprocessing in parallel or at a required timing such as when a call ismade.

According to the present disclosure, it is possible to provide the imagedata alteration detection device, the image data alteration detectionmethod, and the data structure of image data that can easily detect analteration in image data.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. A data alteration detection device, comprising:an alteration detection unit that detects an alteration in image dataincluding data obtained by encoding an image and metadata that is datarelating to the data, wherein the metadata at least includes a firsthash value calculated based on a parameter embedded in a predeterminedarea of the image data in advance, a second hash value calculated basedon a character string of a script for generating a hash value, and thescript, and the alteration detection unit calculates a third hash valuebased on the parameter and the script included in the metadata,calculates a fourth hash value based on the character string of thescript included in the metadata, compares the first hash value includedin the metadata with the third hash value being calculated and thesecond hash value included in the metadata with the fourth hash valuebeing calculated, and detects that the image data is altered when one ofthe values is not equal.
 2. The data alteration detection deviceaccording to claim 1, further comprising an image data reproduction unitthat reproduces an image based on the image data, when an alteration inthe image data is not detected by the alteration detection unit.
 3. Adata alteration detection method comprising: detecting an alteration inimage data including data obtained by encoding an image and metadatathat is data relating to the data, wherein the metadata at leastincludes a first hash value calculated based on a parameter embedded ina predetermined area of the image data in advance, a second hash valuecalculated based on a character string of a script for generating a hashvalue, and the script, the detecting an alteration further comprising:calculating a third hash value based on the parameter and the scriptincluded in the metadata; calculating a fourth hash value based on acharacter string of the script included in the metadata; and comparingthe first hash value included in the metadata with the third hash valuebeing calculated, and the second hash value included in the metadatawith the fourth hash value being calculated, wherein the comparingdetects that the image data is altered when one of the values is notequal.
 4. A data structure of image data, comprising: data obtained byencoding an image; and metadata that is data relating to the data,wherein the metadata at least includes a first hash value calculatedbased on a parameter embedded in a predetermined area of the image datain advance, a second hash value calculated based on a character stringof a script for generating a hash value, and the script.